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Abstract 

We express the joint Renyi entropy of progressively censored order statistics 
in terms of an incomplete integral of the hazard function, and provide a sim- 
ple estimate of the joint Renyi entropy of progressively Typc-II censored data. 
Then we establish a goodness of fit test statistic based on the Renyi Kullback- 
Lciblcr information with the progressively Type-II censored data, and compare 
its performance with the leading test statistic. A Monte Carlo simulation study 
shows that the proposed test statistic shows better powers than the leading test 
statistic against the alternatives with monotone increasing, monotone decreas- 
ing and nonmonotonc hazard functions. 

Keywords: Renyi Entropy, hazard function, Monte Carlo simulation, order 
statistics, Type-II progressively censored data. 
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1 Introduction 

Suppose that X is a random variable having an absolutely continuous distribution 
with density function f(x). The Renyi entropy of order a is denned as 

H a (f) = -r^— log I" {f(x)} a dx a > 0, a + 1, (1.1) 
I - a J_ 00 

and Shannon entropy as H(f) = lim Q ^i H a (f) = — f(x) log f(x)dx provided 
these integrals exist. The Renyi entropy was axiomatized by Renyi [M] and was 
modified by some authors [17]. Renyi entropy has a number of applications in 
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different fields such as statistics pQ , [19] , [21] , [21] , [39] , biomedical engineering [23] , 
statistical mechanics [11], [26], economics [9], stochastic processes [16], [22], [15], 
and some other areas [lOj . [13], [28], [31], [33], [35] . 

The Renyi Kullback-Leibler (R-KL) information in favor of f(x) against g(x) is 
defined as 



and /(/; g) = lmw I a (f; g) = - f(x) log ggdx is the Kullback-Leibler (KL) 
information if both integrals exist. Because I a (f; g) has the property that I a (f; g) > 
0, and the equality holds if and only if / = g, the estimate of the R-KL information 
can be consider as a goodness-of-fit test statistic. 

From the KL information point of view, the problem of goodness of fit test based 
on complete sample studied by some authors including [2] , [T2] and [5B] . Also based 
on censored sample, [32] presented a new test statistic with Type-II censored data 
and [38] modified the previous test statistic. In the case of progressively censored 
data, [6] studied the testing exponentiality based on KL information with progres- 
sively Type-II censored data and goodness of fit test based on KL information for 
progressively Type-II censored data can be found in [T5] . 

Now we are to study the performance of test statistics based on R-KL information 
for progressively Type-II censored data. We provide a new test statistic based on R- 
KL information and compare the power of this test statistic with the performance 
of Shannon KL information, [6]. We show through simulation studies that the 
proposed goodness-of-fit test is more powerful than the test statistic presented in 
[6] for different choices of sample sizes and progressive censoring schemes in nearly 
all cases. However, we can instantly find that this work is not so straight, as we 
need to estimate the joint Renyi entropy of progressively Type-II censored data, via 
a multi-dimensional integral. 

In many life-testing and reliability studies, the complete information may not 
be available on failure times of all experimental units. There are also situations 
where the removal of units prior to failure is pre-planned in order to reduce the 
cost and time associated with testing. For these, and other reasons, progressive 
censoring has been considered, see [29] and [6]. The conventional Type-I, and Type- 
II censoring schemes are not flexible enough to allow removal of units at points 
other than the terminal points of the experiment. So we consider the progressively 
Type-II censoring as a more general censoring scheme. 

The progressive Type-II censoring arises in a life-testing experiment as follows. 
Suppose n units are placed on test. At the time of the first failure, R\ units are 
randomly removed from the n — 1 surviving units. Then at the time of the second 




a>0,a/l 
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failure, R2 units are randomly removed from the n — R\ — 2 surviving units and so 
on. Finally after the mth failure, all remaining R m units are removed. Thus, we 
observe m complete failures and R\+ R2 + ■ ■ ■ + R m items are progressively censored 
from the n units under the test, and so n = m + (Ri + R2 + ■ ■ ■ + Rm)- The vector 
R = (Ri, • • • , Rm) is called the progressive censoring scheme and it is fixed prior to 
the study. A schematic illustration of the progressively Type-II censoring scheme is 
presented as: 



Rl R2 Rm—l Rm 




A schematic presentation for progressively Type-II censored scheme. 



If R = (0, • • • ,0), no withdrawals are made and it corresponds to the complete 
sample situation in which case the usual order statistics will be obtained. If R = 
(0, • • • , 0, n — m), we obtain the conventional Type-II right censoring scheme. We 
will denote the progressively Type-II censored samples as X\- m -n < ^2:m:n < ■ ■ ■ < 
Xm-.m-.n- Progressive censoring scheme, methodology, goodness of fit, estimation 
have been studied in 0], [7j, [5j, [30] and [37]. A book-length account is available in 

The joint Renyi entropy of Xi :m:n , ■ ■ ■ , Xm-.m-.n is simply defined to be 



1 



■i'_>: 



m: n 



Hl-m-.m-.n = ^ _ lo g / • / {/Xi :m: „,- ,X m -.m-.n i X l 5 ' ' ' J %m)Y 



CO 



dxi • • • dx m a > 0, a ^ 1. 

In the above formula, /xi :ro:n ,-,x m:m: n(^i) ■ ■ ■ ,%m) is the joint p.d.f. of all m pro- 
gressively Type-II right censored order statistics (Xi :m:n , • • • , Xm-.m-.n), given in [5] 
as 



fx 1[m[n ,-,X m:m:n (x 1 ,--- ,X m ) = C~[[f(Xi){l - F( Xi )} R > X X < ■■■ < X m 

i=l 

where 

c = n(n - Ri - 1) • • • (n - Ri - R 2 i? m + 1). 

The rest of this paper is arranged as follows: In Section II, we first present H". 
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as a single-integral in terms of the hazard function, h(x), as 



1 m poo 

Ht.m:m:n = ~ log C+— - £ log / /^^^(^{/iCx)} - 1 { 1 - F Xj:m: „ (*) dx, 

" J —oo 

(1.2) 

We provide an estimate of (II. 2D . and define the R-KL information for progressively 
Type-II censored data. In Section 3, we propose a goodness-of-fit test for exponen- 
tiality based on this R-KL information, and use Monte Carlo simulations to evaluate 
the power under different progressive Type-II censoring schemes. We also compared 
the performance of our test statistic with the test presented by [18]. Finally, in 
Section 4, we present an illustrative example. 

2 Renyi Entropy of Progressively Censored Data in Terms 
of Hazard Function 

2.1 Renyi Entropy Representation 

Another expression of (jl.ip is presented in terms of the hazard function as 

#1*1:1 = ~, log / l(x){h{x)y- x {\ - F(x)} a dx. (2.3) 

I- a J_ 00 

We first note that (|2.3p gives a simple expression of Hf. m:n as 

fff! m:n = -logn + - log / f Xl:m .Jx){h(x)r~ 1 {l-F(x)} n ^dx. (2.4) 

1 - a J-oo 

Theorem 2.1 below states that the multiple integral in Hf„, m . m . n can be simplified 
to a single integral. 
Theorem 2.1: 

where 

-I m POO 

H?... m:m .. n = T— "E lo S / /-v.., ,!•'•!{/'(•'•;}" ' {1 " Fx^.Jx)}^ 1 dx. 

1 a _j J— oo 

Proof: See Appendix A. 

The function &" , in Theorem (2.1) can be expressed in terms of log/(x) as 
follows. 
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Lemma 2.1: 



1 ^ / fUam-D 



i=i 



1 dp J 
(1-P) 



a-l 



u=0 V 7 



a-1 



dp 



(2.5) 



where C/(i :7 -_i) is the first uniform order statistic from a sample of size 7^ — 1 and 

7i = m - i + 1 + Xljlj #j for 1 < i < m, c,-_i = Ilu=l 7« for 1 < J < ™, 

°*J = ]ln=i V(7u ~ 7i) for 1 < « < J < m, and 7„ - 7, = 1 for n = i. 
Proof: See Appendix B. 

2.2 Nonparametric Renyi Entropy Estimation 

Now approximating Hi... m:m . n in (I2.5P by 



1 



1 -a 4rf 



> 1 X] (1 



3=1 * u i=l 

estimating the derivative of F~ 1 (p) by 



(1-p) 



a-l 



,«=0 



a-l 



dp, 



where 
and 



E[U i:m:n ] = 1 - 



_ •Ej+w:m:n %j—w.m:n 
Pi+w:m:n Pi—w.m:n 

Pi:m:n — -^[-f\^Q:m:n)] — [t^i:m:n] > 

j + Rm-j+l + • • • + Rm 



j + l + Rm-j+l + ---+Rr, 



j =m -i+l w ■ - ■ '-"<■-]- 

and approximating the integral of J Q g(x)dx by a Riemann sum by eg(§), we obtain 



an estimate of Hf... m . m . n as 



1 



H a {w,n,m) ^E^^VE ,., T )a -x 

i=i 'tl 1 27^ 



^■■■m-.m-.n ^ s obtained e 
n (ui, re, m) = — log c + H a (w, n, m). 



,u=0 

Thus, from Theorem 2.1, an estimate of Hf„, m , m . n is obtained as 

l---m:m 



a-l 



5 



2.3 Renyi Kullback-Leibler Information, and Test Statistic 

For a null density function f°(x; 9), the R-KL information from progressively Type- 
II censored data is denned to be 

-1 POD fX2:m:n 

A a ..~(/;/°) = — riog / ■■■ / 

{fx 

l:m:ni'" i^mimin 

(xi, ■ ■ ■ ,x m ;0)} c 



{fx 1:m:n ,:. ,X m:m:n ( X l' " ' ) X m] 0)Y 



jdxi ■ ■ ■ dx r 



Consequently, the R-KL information can be estimated by 



Il:. m :m:n(f;f°) = ~ ^log f (xf, 9) - J^log(l - F°(xj]9)). (2.6) 

i=i j=i 
Thus, the test statistic based on /"„ mm (/; f°)/ n is given by 



T a (w,n,m) = H a (w,n,m) 

n n 



where 9 is an estimator of 9. 



£ log f( Xj ; 9) + £ Rj Iog(l - F°( Xj ; 9)) 

3=1 3=1 

(2.7) 



3 Testing Exponentiality Based on The Renyi Kullback- 
Leibler Information 

3.1 Test Statistic 

Suppose that we are interested in a goodness of fit test for 

tf :/ (x) = iexp(-f), 



H A :f(x)^±exp(- 

where 9 is unknown. Then the R-KL information for progressively Type-II censored 
data can be approximated, in view of (|2.6|) . with 

1 m 

I?- — (/: /°) = -flt.n,rr,n + ™ log 9 + - ^(Rj + l)X j:mM . 

3=1 

If we estimate the unknown parameter 9 by the maximum likelihood estimate, 
{J2T = i(Rj + l)Xj :m:n )/m, then we have an estimate of Ii... m:m:n (f', f°)/n as 



T a (w,n,m) = H a (w,n,m) + — 

n n 



(3.8) 
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Under the null hypothesis, T a (w,n,m) for a close to 1, will be close to 0, and 
therefor large values of T a (w,n,m) will lead to the rejection of Hq. 

3.2 Implementation of the Test 

Because the sampling distribution of T a (w,n,m) is intractable, we determine the 
percentage points using 10,000 Monte Carlo simulations from an exponential dis- 
tribution. In determining the window size w which depends on n, m, a and 77, 
significance level, we chose the optimal window size w to be one which gives the 
minimum critical points in the sense of |12| . Also, we chose a in R-KL information 
using the trial and error method. However, we find from the simulated percentage 
points that the optimal window size w varies much according to m rather than n, 
and does not vary much according to a, if a < 1. In view of these observations, our 
recommended values of w for different m are as given in [32] and our recommended 
value of a, using the trial and error method, is a = 0.4. 

To obtain the critical values, after deciding about the value of w and a, simulate 
the whole procedure by taking the observation from the E(l) distribution, and 
calculate the value of T a (w,n,m), for 10,000 times. 

3.3 Power Results 

There are lots of test statistics for exponentiality concerning uncensored data [3], 
[H] j |2Q] , [25] , [27] , but only some of them can be extended to the censored data. 
We consider here the test statistic [6] among them. [6] proposed the test statistic as 

T(w, n, m) = —H(w, n,m) H 

n 

where 

TT / \ J- , / %i+w:m:n %i— w.m:n \ /-. 777, \ / 77l\ 

H(w, n, m) = - > log — r — - 1 log 1 . 

n ~[ \E{Ui+w.mm) - E \Pi-w:m:n) J V nJ V nJ 

As the proposed test statistic is related to the hazard function of the distribution, 
we consider the alternatives according to the type of hazard function as follows. 

• I) Monotone increasing hazard: Gamma and Weibull (shape parameter 2), 

• II) Monotone decreasing hazard: Gamma and Weibull (shape parameter 0.5), 

• III) Nonmonotone hazard: Center Beta (shape parameter 0.5), Log normal 
(shape parameter 1). 
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Table 1: Power comparison for different hazard alternatives at f0% significance level for several 
progressively censored samples with the sample size is n = 10 
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5 


0,0,0,5,0 


T(w, n, m) 

\ 7 7 f 


0.312 


0.469 


0.023 


0.031 


0.032 


0.224 






T a (w, n, m) 


0.365 


0.549 


0.038 


0.039 


0.045 


0.274 


5 


0,0,0,0,5 


T(w, n, m) 


0.323 


0.440 


0.033 


0.038 


0.035 


0.255 






T a (w, n, m) 


0.370 


0.516 


0.044 


0.046 


0.047 


0.305 


8 


2,0,0,0,0,0,0,0 


T(w, n, m) 


0.492 


0.807 


0.018 


0.055 


0.066 


0.189 






T a (w, n, m) 


0.538 


0.843 


0.024 


0.067 


0.086 


0.229 


8 


0,2,0,0,0,0,0,0 


T(w, n, m) 


0.488 


0.812 


0.017 


0.054 


0.063 


0.187 






T a (w, n, m) 


0.572 


0.863 


0.022 


0.065 
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0.225 


8 


1,0,0,0,0,0,0,1 


T(w, n, m) 


0.470 


0.754 


0.023 


0.060 


0.042 


0.230 






T a (w, n, m) 


0.526 


0.803 


0.031 


0.075 


0.053 


0.275 


8 


0,0,0,0,0,0,2,0 


T(w, n, m) 


0.406 


0.684 


0.021 


0.040 


0.042 


0.211 






T a (w, n, m) 


0.457 


0.730 


0.027 


0.050 


0.053 


0.241 


8 


0,0,0,0,0,0,0,2 


T(w, n, m) 


0.453 


0.713 


0.026 


0.064 


0.035 


0.247 






T a (w, n, m) 


0.511 


0.768 


0.036 


0.080 


0.048 


0.310 



We used 10, 000 Monte Carlo simulations for n = 10, 20 and 30 to estimate the 
power of our proposed test statistic, and the competing test statistic. The simulation 
results are summarized in Figures 1-3, and Tables 1-3. 

4 Illustrative Example 

In this section, we present an example to illustrate the use of the test statistic 
T a (w,n,m) for testing the validity of the exponential distribution for an observed 
progressively Type-II right censored sample. We consider the data of Nelson [29], 
Table 6.1, concerning times to breakdown of an insulating fluid tested at 34 kilo- 
volts. Viveros and Balakrishnan [37] used Nelson's data, and randomly generated a 
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Table 2: Power comparison for different hazard alternatives at 10% significance level for several 

progressively censored samples with the sample size is n = 20 
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0.337 


0.479 


0.043 


0.048 


0.045 


0.311 


5 


0,0,0,0,15 


T(w, 77, 777) 


0.332 


0.421 


0.035 


0.037 


0.040 


0.356 






T a (w, 77, 777) 


0.376 


0.475 


0.044 


0.044 


0.044 


0.400 


10 


10,0,0,..., 0,0,0 


T(w, 77, 777) 


0.632 


0.930 


0.006 


0.043 


0.048 


0.266 






T a (w, 77, 777) 


0.675 


0.935 


0.060 


0.226 


0.094 


0.360 


10 


0,10,0,..., 0,0,0 


T(w, 77, 777) 


0.672 


0.946 
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0.039 


0.026 


0.280 






T a (w, 77, 777) 
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0.948 
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0.231 
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10 


1,1,1,..., 1,1,1 


T(w, 77, 777) 


0.605 


0.865 
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0.036 


0.012 


0.357 






T a (w, 77, 777) 


0.630 


0.875 
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0.060 


0.443 


10 


0,0,0,... ,0,10,0 


T(iu, 77, 777) 


0.376 


0.588 


0.026 


0.030 


0.028 


0.267 






T a (w, 77, 777) 


0.404 


0.623 


0.079 


0.126 


0.075 


0.301 


10 


0,0,0,... ,0,0,10 


T(w, 77, 777) 


0.534 


0.735 


0.037 


0.077 


0.028 


0.473 






T a (w, 77, 777) 


0.584 


0.779 


0.117 


0.197 


0.085 


0.522 
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Table 3: Power comparison for different hazard alternatives at 10% significance level for several 

progressively censored samples with the sample size is n = 30 
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Fi gU.16 ll Power comparison in monotone increasing hazard alternatives (Gamma: shape 2 (a, c, e), 
Weibull: shape 2 (b, d, f)) at 10% significance level for several progressively censored samples when the 
sample size is 10 (a, b), 20 (c, d), and 30 (e, f). 
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Figure 2: Power comparison in monotone decreasing hazard alternatives (Gamma: shape 0.5 (a, c, e), 
Weibull: shape 0.5 (b, d, f)) at 10% significance level for several progressively censored samples when the 
sample size is 10 (a, b), 20 (c, d), and 30 (e, f). 
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Figure 3: Power comparison in nonmonotone hazard alternatives (Beta: shape 0.5 (a, c, e), Log-Normal: 
shape 1 (b, d, f)) at 10% significance level for several progressively censored samples when the sample size 
is 10 (a, b), 20 (c, d), and 30 (e, f). 



Table 4: Nelson's data and progressively Type-II censoring scheme 
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progressively Type-II censored sample of size m = 8 from n = 19 observations. The 
data, and the progressive censoring scheme employed are given in Table 4. 
The test statistic computed from (13.8|) is 

Tfl 

T a (w,n,m) = H a (w,n,m) -\ 

n n 

where w = 3 according to Table I in [32]. We estimate the unknown 9 by the 
maximum likelihood estimator, and so the null distribution can be approximated by 
an exponential distribution with 6 = 9.09. The p-value is then computed as 

P (r a (w,n,m) > 0.2422|tf : fx(x) = ^ exp (7^)) = °- 9737 

which provides very strong evidence that the observed progressively Type-II cen- 
sored sample is from an exponential distribution. 



lQ g f^E^+^-nj +1 



0.2422, 
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5 Conclusion 



In Renyi entropy, with controlling a about 1, we provided a test statistic which 
was quite powerful when compared to an existing goodness-of-fit test proposed for 
progressively Type-II censored data due to [6]. Because the sampling distributions 
of these test statistics were intractable, we determined the percentage points using 
10, 000 Monte Carlo samples from an exponential distribution. 

We saw from the Figures 1-3 and Tables 1-3 that the proposed test statistic 
showed better powers than the competing test statistics against the alternatives 
with monotone increasing, monotone decreasing and nonmonotone hazard functions, 
which apply to many real-life applications. Also, we saw from Figures 1-3 and Tables 
1-3 that the scheme = n — m, R2 = 0, . . . , R m = 0) showed higher power than 
the other schemes when the alternative was a monotone increasing hazard function. 
For the alternative with monotone decreasing hazard functions, the scheme (R\ = 
0, . . . , Rm-i = 0, R m = n — m) (the conventional Type-II censored data) showed 
higher power. Finally, for the alternative with a nonmonotone hazard function, 
sometimes the former censoring scheme gave a higher power, and sometimes the 
latter censoring scheme did. 

This work has the potential to be applied in the context of censored data and 
goodness of fit tests. This paper can elaborate further researches by extending such 
modifications for other censoring schemes. Also, this area of research can be ex- 
panded by considering other distributions besides the exponential distribution such 
as Pareto, Log-Normal and Weibull distributions. Finally, the method of a deter- 
mination can be developed by considering an optimality algorithm which maximize 
the power. 

Appendix A 

Proof of Theorem 2. 1 

By the decomposition property of the entropy measure in [15] . we have 

11 1— m:m:n l:m:n ' 2\\:m:n ' ' ' ' r\l — l:m:n ' ' ' m\m—l:m:n' 

The conditional p.d.f of progressively Type-II censored order statistics is given in 
[30] as 

fx \x 1 (x r \x r -i) 

where h{x r ) = 1 _^ r ^ n and so fx r . m . n \x r _ 1 . m . n ( x r\x r ~i) can be interpreted as the 
density of the first order statistic among an n — Y^j=i Rj — r + 1 sample from 



/ ^ \ r 

= \n - Rj ;- r + 1 h(x r ) 
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i-f(x i) ' so we can °btain by ([27 

(r-l 
n "X^' " r + 1 

+- log / fx^JxnHx^il-Fx^Jx^dx. 

I- a J_ 00 

The required result then follows. 

Appendix B 

Proof of Lemma 2. 1 

The p.d.f of X T:m - n is given in [5] as 

fx j:m .Jx) = Cj-tfix) J>y(l - FCx))^" 1 , 
i=l 

where 7$ = m — i + 1 + X^j=i for 1 < i < ?n, Cj_i = n«=i 7« f° r 1 < J < 
°*J = Ilu=l 1 /(7« - 7i) for 1 < i < j < m, and 7u - 7i = 1 for u = i. So Hf... m . m . n 
is equal to 

1 m />oo 

l ~ a ~[ J -00 

1 m /"OO J 



0-1 



dx 



— £ log £ / Ci _!/(X) £ OyM"- 1 { 1 " /%, m: „ (X)}^ 1 dx 

— log £ ^ y ^ ^/(x) g ay 



3=1 



j-i / \ 

X)H{F(x)Hl-F(x)}»-» 

.«=o 



a-l 



dx 



m / f 



J 



Edp } 



{ 



dF-^phi- 



3=1 

Hence, the lemma is proved. 



i=i 



(1-p) 



0-1 



x)} 

[Put: F(s) = p] 



0-1 



dp 
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